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ONLINE SIMULTANEOUS/ALTERNATING - AUDIO/VIDEO/VOICE DATA 
BASED SERVICE AND SUPPORT FOR COMPUTER SYSTEMS 



FIELD OF THE INVENTION 

This invention relates generally to online support services for computer users and 
specifically to an online support service using integrated audio, image and other data to provide 
5 service and support functions. 

BACKGROUND OF THE INVENTION 

The popularity of personal computers has made service and support for such 
computers critical to insure consumers' efficient use of the computers. Further, because computer 
designs are so similar, service and support programs also serve to differentiate different 

10 manufacturers of the computers. A superior service and support program may well be the 
difference between mediocre and superior personal computer sales. 

However, even though a good service and support system is important to a 
computer manufacturer, such a program is difficult and expensive to maintain because of the 
complexity of today's computer systems and related components, the lack of technical expertise 

15 at the user end and the difficulty of solving technical computer problems from a remote location. 
Part of the difficulty also arises from the "open architecture" of a typical personal computer. Such 
an open architecture allows many different manufacturers to produce hardware components and 
software applications for a consumer's personal computer. The combination of specific hardware 
and software residing on, or operating within, a computer system is referred to as the particular 

20 "configuration" of that computer system. Because several different hardware and software 
manufacturers contribute to any particular configuration it is difficult for a single manufacturer 
to provide adequate support for all of the components in the configuration. Also, a given 
configuration will likely change every few months, or so, as the user purchases new or upgraded 
hardware and software components. 

25 Despite the difficulty in providing a single source of support for a user's 

configuration, the user has a very strong expectation of being able to go to the original equipment 
manufacturer (known as an "OEM," i.e., the manufacturer or assembler of the basic computer 
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components initially shipped together as a "system," typically including the motherboard and 
housing, hard disk, monitor, operating system software and bundled applications) for solutions 
to any problems that the user experiences with their configuration and is disappointed if such a 
service is not provided. 

5 Today's typical support programs allow a user to contact a customer service 

center staffed and run by the OEM of the user's computer. The contact is usually by telephone. 
Often the user can use the Internet, modem or other means to obtain digital information or 
components such as an updated "patch" to fix a software problem, or instructional text on how 
to solve a problem. However, these methods of communication and information access provided 

10 by a typical support program do not provide adequate help to the user for various reasons. 

One reason that traditional support methods fall short is that in order to properly 
diagnose a user's computer system problems a customer service representative in a remote 
location needs to obtain detailed and accurate information about the user's configuration. Often 
this is complicated because of nonstandard provision of information by each of the hardware or 

15 software component manufacturers as to identifying their product, and as to describing and 
implementing the way their product interfaces into the overall computer system. 

Also, a personal computer user is generally not a technical person. In this respect, 
it is difficult for a customer service representative at a remote location to obtain the proper 
information from the user, or to get the user to perform tasks on the computer system that are 

2 0 necessary in order to fix a problem with the computer. Even if the user is technically trained, it 

is often difficult to communicate adequately with the user over standard communication devices 
such as the telephone, fax machine, mail correspondence, etc. Yet other problems exist with 
typical online service and support practices today. For example, companies will often provide 
"patches" on the Internet or World Wide Web ("WWW 1 or "WEB") which the user must 
25 download and install in order to proceed with fixing problems on their computer. This process 
is often time consuming and is usually handled by having the user end the conversation with the 
customer service representative ("customer service rep"), perform the download, install the patch, 
test out the software, and call the customer rep back, if necessary. Since customer service centers 
are notoriously overloaded, the act of obtaining another session with a customer rep is frustrating 

3 o and requires a considerable amount of time. 
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Thus, it is desirable to provide an online service and support system for improved 
support, diagnosis and servicing of remotely located personal computers. 

SUMMARY OF THE INVENTION 

The present invention connects a remotely located customer service and support 
5 center with a user's personal computer. 

One embodiment of the invention is an online computer support system including 
a user computer coupled to a communication line. The online computer support system also 
includes a customer service center including a customer representative and electronic equipment 
coupled to the communication line. The user computer and the electronic equipment each include 
10 means for transferring audio information over the communication line; means for transferring 
image (including video) information over the communication line; and means for transferring 
digital data over the communication line. The transfer protocol can allow for either simultaneous 
or alternating transmission of the different data types. 

Another embodiment of the invention provides an online computer support system 
15 including a user computer coupled to a communication line; hardware components coupled to the 
user computer; a customer service center including a customer representation and electronic 
equipment coupled to the communication line; means, coupled to the user computer and to the 
electronic equipment, for transferring digital data over the communication line; and means for 
transferring information about the hardware from the user computer to the customer service 
2 0 center over the communication line. 

Since another embodiment of the invention provides an online computer support 
system including a user computer coupled to a communication line, software residing in the user 
computer, a customer service center including a customer representation and electronic equipment 
coupled to the communication line; means, coupled to the user computer and to the electronic 
25 equipment, for transferring digital data over the communication line; and means for transferring 
information about the software from the user computer to the customer service center over the 
communication line. 

BRIEF DESCRIPTION OF THE DRAWINGS 
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Fig. 1 is a system-level illustration of the online customer service system of the 
present invention; 

Fig. 2 shows a typical computer system suitable for use with the present invention; 

Fig. 3 shows basic subsystems in a typical computer system; 
5 Fig. 4 shows a dialog box of the user interface in the present invention; 

Fig. 5 shows a screen image of the VAIO Space navigator along with an SOS icon 
for launching the online support system of the present invention; 

Fig. 6 shows an initial window presented to the user after a connection with the 
customer service center is made; 
10 Fig. 7 is a first screen display of a window viewed by the customer service 

representative during a session with a user; 

Fig. 8 shows a second screen display viewed by the customer service 
representative; and 

Fig, 9 is a diagram of interrogation software in a user computer. 

15 DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT 

Fig. 1 is a system-level illustration of the online customer service system 100 of 
the present invention. In Fig, 1, customer service center 102 includes electronic equipment such 
as multimedia server 108, external storage devices 104 and 106, computer stations 1 10 and 1 12 t 
and customer service reps 1 14 and 1 16. Fig. 1 shows multimedia server 108 in communication 

20 with customer site 120 via a communication link 122, In a preferred embodiment, voice, image 
and other digital data information are transferred over a so-called "plain old telephone system" 
(POTS) line. Other communication lines may be used such as an Integrated Services Digital 
Network (ISDN) line, Tl line, coaxial cable, fiber optic cable, satellite transmission, cellular 
telephone network, or other connection. 

25 Simultaneous transfer of voice and digital data (including images, video, text, etc.) 

is achieved with a protocol such as Digitat Simultaneous Voice/Data (DSVD). Alternating 
transmission of different data types can be achieved by using, for example, the Asynchronous 
Voice/Data (AVD) protocol. The preferred embodiment uses both of these protocols. Other 
protocols may be used. 
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In Fig. 1, other customer sites (not shown) may be simultaneously supported. 
Naturally, additional computers, computer peripherals, customer service reps and electronic 
equipment may be included at the customer service center. For example, a video server, 
teleconferencing capability, etc. may reside at the customer service center and be coupled to the 
5 communication link via multimedia server 108. Other manner of communicating with customer 
site 120 over a communication link may be possible, such as by directly linking a computer, such 
as computer 1 10 at customer service center 102, to customer site 120. 

At customer site 120 are the user computer 130 and user 132. Various types of 
information can be captured and digitized at customer service center 102 for transfer to a 

10 customer site, such as customer site 120, over the communication line. In Fig. 1, customer 
service reps are shown with audio head sets for digitizing voice via a microphone such as 
microphone 160 and for playing back sound through earpiece 162. Video camera 152 is used to 
capture images. The images are then digitized. The overall image and audio data can be 
synchronized into a presentation, or video program, for playback at the customer site. Other 

15 types of information may be exchanged with the user as described below. 

At customer site 120 the user's computer 130 is, preferably, a multimedia 
computer. That is, it is preferable for the user to have speakers such as speaker 164, and a 
microphone such as microphone 166 connected to user computer 130 to provide for the 
transmission and playback of audio information. As computers are becoming more popular for 

2 0 educational, entertainment and other home uses, they are increasingly being provided with 
peripherals such as speakers and microphones as standard equipment to allow playback and 
capture of audio information. Additionally, it is likely that the use of digitizing cameras will gain 
in popularity as the cost for such cameras and digitizing hardware decreases. If the user is 
equipped with a way to capture and transfer image information back to customer service center 

25 102 a future embodiment of the invention will make use of such added capability. 

An alternative way for the user to transfer audio information with customer service 
center 102 is by using a standard telephone handset 168 shown at customer site 120 in Fig. 1. 
The preferred embodiment uses a personal computer manufactured by Sony Corporation which 
is provided with a jack that is compatible with the standard telephone line. Thus, the user may 

30 plug their standard telephone into the provided jack and establish voice communication with 
customer service center 102 by using the telephone. The preferred embodiment uses a standard 



5 



WO 97/49030 



PCT/US97/09320 



telephone line for simultaneous transfer of image, audio, and other digital data with a standard 
modem at 28.8 kbps. The data format is Digital Simultaneous Voice/Data (DSVD) protocol 
developed by, among others, Intel Corporation. 

Simultaneous transfer of data using DSVD has the advantage that data can be 
5 exchanged during a telephone conversation so that, for example, an image frame, which consists 
of a large amount of data, can be sent over the line while the customer service rep and the user 
are having a conversation. At other times it may be more advantageous to use the line exclusively 
for voice or data transfer as in the case where a large amount of image data must be sent before 
the support session can continue. In this latter case, an Asynchronous Voice/Data (AVD) 

10 implementation may be used. The preferred embodiment uses a protocol implementation called 
VOICEVIEW, developed by Radish Corporation. This allows the customer service rep and the 
user to switch back and forth between voice and data transfers as often as necessary. 

Fig. 2 shows a typical computer system 180. Such a computer system is 
representative of the type that may be encountered at a customer site, for example, as computer 

15 130 of Fig. 1 . Also, such a computer is typical of the type that can be used at customer service 
center 102 (of Fig. 1) as workstation computers for customer reps, such as computers 1 10 and 
112. 

In Fig. 2, computer system 180 includes monitor 182 having display screen 184 
and housing 186. Housing 186 typically includes standard computer components such as a hard 
2 O disk drive, random access memory (RAM), processor, and additional hardware components (not 
shown). Computer system 180 also includes user input devices such as keyboard 188 and mouse 
190. Mouse 190 has one or more buttons such as those shown at 192. 

Fig. 3 shows basic subsystems in a typical computer system such as the computer 
system of Fig. 2. 

2 5 Fig. 3 shows system bus 200 used to allow transfer of information between 

subsystems, or components, connected to the bus. For example, central processor 206 executes 
programs residing in system memory 204 or on fixed disk 218. Also, information can be 
transferred to devices such as printer 208, serial port 214, I/O controller 202 or external interface 
220. Visual information is sent to display adapter 210 for display on monitor 212. Data 

3 0 generated by a human user of the computer is received through input devices such as keyboard 

216 or a mouse pointer device (not shown). Other devices may be connected to the system bus 
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through I/O controller 202 or serial port 214. For example, I/O controller 202 can be a Small 
Computer Systems Interconnect (SCSI) controller or an IEEE 1394 standard bus interface. Many 
standard devices such as disk drives, video capture cards and consumer electronic devices may 
be connected to these types of interfaces. Serial port 214 can be used to connect the computer 
5 to a telephone line or other communication link, printer, mouse or other peripheral.. 

Fig. 3 is merely illustrative of some typical subsystems, or components, found in 
a computer system, The configuration of any given computer system may vary greatly from that 
shown in Fig. 3. For example, some computer systems will not have a printer or external interface 
adapter. Other devices that are common to computers, such as a sound card, are not shown as 
10 in Fig, 3. Typically, the speakers and microphone at the user computer as shown in Fig. 1 
communicate with the processor and communication line through a sound card mated directly to 
the bus. or through an interface controller such as a SCSI controller. 

In general, subsystems may be added to, or taken away from, those shown in Fig. 
3 without departing from the scope of the present invention. 

15 

Using Audio. Image And Other Data Over A Single Communication Line To Provide 
Improved Support 

The present invention provides a user, or customer, with both audio and image 
information, along with other types of information, to improve customer service, product 
20 registration, analyzing, repairing and updating hardware and software components of the user's 
system. A simple and immediate benefit of providing audio and image data to the user during a 
customer service session is that the user can see what the customer service rep looks like. This 
allows the user to relate more easily to the customer service rep and puts the user in a good frame 
of mind to deal with the technical problems and frustration associated with having the computer 

2 5 malfunction. 

Fig. 4 shows a first screen shot of the user interface used in the customer service 
system of the present invention. This screen appears when the user attempts to connect for a first 
support session with the customer service center. The user can, alternatively, invoke the session 
by clicking on an icon that is available from within the application "navigator" provided with each 

3 0 system sent by the manufacturer. In the preferred embodiment, the manufacturer is Sony 

Corporation, the navigator is referred to as "VAIO Space." 
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Fig. 5 shows a screen image 290 of the VAIO Space navigator in the preferred 
embodiment along with an SOS icon 292 used to launch the Sony Online Support system of the 
present invention. The VAIO Space navigator per se does not form an essential part of the 
present application. 

5 Returning to Fig. 4, in the dialog box shown in Fig. 4 the user enters their name, 

phone number and a description of the problem they are having. After the information is filled 
in, "SOS Help" can be invoked by depressing the button to access a local help data base residing 
on the user's computer. Alternatively, the "Call SOS" button can be pressed to dial up the remote 
customer service center for assistance from a customer service rep. Note the instructions under 

10 the buttons which direct the user to connect a telephone to the PC if the user does not have 
speakers and a microphone already attached to their PC. After the user presses the "call SOS" 
button, and after the customer service rep answers the call, the user is presented with new 
windows shown in Fig. 6. 

Fig. 6 shows an initial window presented to the user after a connection with the 

15 customer service center is made. Additionally, a "dial up" window is shown. The dial-up window 
is used to change the telephone number for the customer service center, if necessary. 

In Fig. 6, the name, phone number, extension, and visual image of the customer 
service rep who will be helping the user in the current support session is shown. The user has 
options of e.g., saving, printing, or annotating the information shown in the initial window 300. 

20 For example, the user can save the text and image information to disk under a chosen file name 
by depressing "SAVE AS..." button 308. Before saving, the user can edit, or annotate, the 
information by depressing button 3 10. "SEND" button 302 is used to forward a screen image 
such as the screen image shown in Fig. 6, to another PC user assuming that the other PC user has 
a DSVD modem and SOS software. The user can print the text and image to a printer by 

25 depressing the "PRINT" button 306. "COPY" button 304 copies the screen image to the local 
clipboard of the operating system to allow the image to be transferred to another application. The 
user may close the initial screen by depressing the "CLOSE VIEWER" button 312. If the view 
is closed, image data is no longer displayed. However, the user can still be in voice and data 
contact with the customer service rep. 

3 0 The user may return to the SOS dialog box shown in Fig. 4 by depressing the 

"SOS Screen" button 314. 
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The user may return to the SOS dialog box shown in Fig. 4 by depressing the 
"SOS Screen" button 314. 

In a preferred embodiment of the invention a single-frame picture of the customer 
rep will be shown in the image area at 316. In future embodiments, as communication line 
5 bandwidth increases, the image at 3 16 can be enlarged to fill the entire window, or display screen, 
as desired, and can be animated to a fiill-motion video by having the image updated at several 
frames per second. Also, synchronized audio will be provided so that the user can see and hear 
the customer service rep in real time. Not only will this make the support session more human 
and friendly, but it will also facilitate the exchange of information by, for example, allowing the 
10 customer service rep to use hand gestures, hold up different objects or diagrams in front of the 
video camera, play instructional videos or tutorials, etc. The user may selectively capture screens 
of the images for future reference or may capture entire segments of the video. Also, the user 
may annotate the frames of video by depressing the EDIT button and typing in additional 
informational "notes." 

15 In the future, personal computers will be more apt to have video cameras at the 

user end. Another option will be camcorders connected to a PC by using IEEE 1394 standard 
serial data buses. This will allow the customer service rep to view the user, and objects or 
documents the user might have. 

Fig. 7 is a first screen shot of a window viewed by the customer service rep during 

2 0 a session with a user. In Fig. 7 a box at the center of the window shows a log of session 
highlights and basic information relevant to the session. For example, the agent's name is entered 
into the log as the first entry at the top of the box, The second entry shows that user information 
has been obtained from the user successfully. That is, the information which the user has entered 
into the dialog box of Fig. 4 is obtained. As discussed below, additional information is transferred 

2 5 at the initial connection between the user and the customer service rep. Finally, the fact that an 
acknowledgment has been sent back to the user computer is logged. As discussed below, 
identification information obtained from the user's computer is used to index a data base at the 
customer service center to include detailed information about the history of support activity for 
a specific user. 

30 In Fig. 7, the box labeled "notes:" shows information retrieved from the data base. 

As can be seen, the data base includes information that this is the second call from this customer. 
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and that the customer is having a problem with his CD caddy. Information such as the 
information that this is the second call from the customer is generated automatically by the 
database or other record-keeping features of the customer service center. Additional information, 
such as the information that the customer's problem in a prior session was with the CD caddy, 
5 is entered by the customer service rep and maintained in a database at the customer service center. 

The box to the right of the window shown in Fig. 7 contains a "Macro List" which 
allows the customer service rep to select an activity, text, etc. with a single mouse click to more 
quickly provide the user with information or to perform a function. The macros in the list can also 
be selected by using predesignated buttons or hot keys. 

10 Some of the functions that the customer service rep may select include predefined 

tests that can be sent to the user's computer and executed at the user computer. In a preferred 
embodiment, the code for these tests already exist at the user computer since it was provided at 
the time of initial shipment of the user's computer to the user. Other common tasks such as 
getting a directory listing of the user's root directory, getting copies of the autoexec.bat and 

15 config.sys files, getting predesignated user info or obtaining copies of other critical documents or 
files in the user system may be instantly obtained by using the Macro List. In the preferred 
embodiment transfer of data (e.g., files such as the autoexec.bat and config.sys files) takes place 
in the background of the session so that the user and customer service rep can communicate both 
through audio and image data while a file transfer is going on. 

20 Fig. 8 shows a second screen display viewed by the customer service rep. 

In Fig. 8, the customer service rep has selected viewing of the file 
USERINFO.FMP Part of the information in USERINFO.FMP is provided by the user while other 
of the information is obtained automatically by software executing at the user's computer. For 
example, in Fig. 8 the user's name and phone number appear along with answers to queries 

25 provided by software at the user end that interrogates hardware and software components in the 
user's system. Part of the interrogation automatically provides the customer service rep with 
information as to the user's hardware and software components and any existing upgrades to 
these components. 

In the preferred embodiment, a standard 28.8 kbps modem is used to transfer data 
30 over the telephone line. Of this bandwidth, approximately 9600 bps is used for voice 
communications and about 14.4 kbps is used for data transfer, including video and image transfer. 
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The remaining bandwidth is used for communication protocol and overhead. Other dynamic and 
static allocations of data are possible. 

One advantage of the present invention is that it allows a customer service rep to 
instantly present ready-made instructions such as text or video instructions to the user. Thus, 
5 when the customer service rep determines that the user's problem is, for example, that the user 
needs to know how to configure their new peripheral, the customer service rep can send the 
appropriate text or video tutorial to the user. The selection of the appropriate tutorial can be, for 
example, a selection on the customer service rep's Macro List. Such tutorials can be stored on 
a hard disk or Compact Disk - Read Only Memory (CD-ROM) disk or other media. For example, 

10 in Fig. 1, a server database including the tutorial can be connected to the network at the customer 
service center in a manner similar to that for connecting external storage devices 104 or 106. 

Other advantages are provided by the present invention. For example, the 
customer service rep can "talk" over a ready-made tutorial as the tutorial is presented. Either the 
customer service rep or the user may stop and start the tutorial, or fast forward and rewind the 

15 tutorial similar to a videotape. Also, both parties may "point" to items shown on the screen. That 
is, both the user and customer service rep's mouse pointers can appear on each computer screen. 
This facilitates communication. Text that is typed in by one party also appears on the other's 
screen and can be saved by taking a "snapshot" of the screen at any time or by recording a series 
of screens, as in a video. Drawing, by using a mouse and pointer in a manner similar to standard 

2 0 drawing programs is possible. This allows, for example, the customer service rep to circle or 
otherwise highlight and indicate objects on the user's screen. 

A customer service rep can also bring in additional support reps to help in the 
user's support session. For example, customer service rep 114 of Fig. 1 may transfer the user's 
session to customer service rep 1 12. Thus, customer service rep 112 may take over a session in 

2 5 progress. Alternatively, customer service rep 1 12 can "join" in to a session already being handled 
by customer service rep 1 14. Text and pointer movement for this second customer service rep 
could appear in a different color on the user computer's screen than the color for the first 
customer service rep. The voice and image data from customer service rep 112 are provided in 
a similar manner as discussed above. 

30 

Reading Hardware and Software Properties 

11 
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Another aspect of the present invention is the ability to read properties of the 
hardware and software components in the user computer system. The present invention uses 
"interrogation software" comprising the Windows Registry Browser (part of the operating system 
provided by Microsoft Corp.), the Desktop Management Interface (DMI) browser (provided by 
5 Intel Corp. with modifications by Sony Corporation) including PC_INFO.EXE, and software 
developed by Sony Corporation creating executables such as "register.exe" to query and obtain 
information about hardware and software components in the user computer. Examples of the 
types of information that the interrogation software can obtain are size of a software data file or 
program, who created the software module, when the software or hardware component was 

10 installed, currently installed Dynamic Link Libraries (DLLs), the version number of the hardware 
or software component, Interrupt Request (IRQ) levels assigned to a hardware component, device 
drivers used by a hardware component, status of a component, memory used or allocated, etc. 
This information forms the heart of most problems that a user can experience. 

Fig. 9 shows a diagram of interrogation software 342 residing in user computer 

15 340. Interrogation software 342 communicates with the resident operating system, the DMI and 
hardware components in the user's computer system. 

Most of the information that the interrogation software obtains is from the DMI 
and some operating system data sources, such as the Windows Registry for the Windows 95 
operating system. The DMI provides low-level communication with hardware devices to obtain 

20 information on properties of the hardware devices, or components. The information obtained is 
transferred to the Customer Service Center via the communication link as described above. 

Table I shows the types of information that are transferred from the interrogation 
software to the Customer Service Center. In Table 1, the interrogation software provides the 
information listed after the line U ;LPT - Printer Ports." All information above this line, under the 

25 headings ";ComponentID Group" and ";Base System" is provided in non-volatile memory as 
programmed at the manufacturing factory of the computer system, discussed below in the next 
section. 

Thus, the provision for interrogation software in the present invention provides 
the customer service rep with timely, complete and accurate information as to the user's 
30 configuration of hardware and software. Such information is critical to the efficient solving of a 
user's computer problems. 
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Factory-Stored Identifiers 

Another feature of the present invention is the use of factory-stored identifiers to 
provide necessary information to tie-in a particular user's computer to database statistics and 
other information kept at the Customer Service Center. In the preferred embodiment, the 
information programmed at the factory includes the information in Table I's first two sections 
labeled ";ComponentID Group" and ";Base System". For example, the manufacturer ("Sony 
Corporation" in the preferred embodiment) and serial number are programmed before the 
computer is shipped to an end-user or to a retailer 

This information is transferred to the Customer Service Center by the interrogation 
software at the start of a support session. At the Customer Service Center, information such as 
the serial number is used to index a local database to retrieve records for the user's particular 
computer such as a history of support sessions. By embedding the information directly into non- 
volatile memory on the motherboard of the computer system, e.g., in a read-only memory (ROM) 
chip, and by transferring the information at the start of each support session, the accuracy of such 
data and tracking of user problems is ensured. 

The information provided by the interrogation software, in conjunction with 
records kept on the local database, assists the customer service rep in determining copyright 
protection issues, validating warranties, inventory control, maintaining service and upgrade history 
and in manufacturing process flow control. 

Online Agents and Other Features 

The architecture of the present invention allows a customer service rep to transfer 
software "agents" to the user computer system. The transfer of an agent is similar to the transfer 
of other data such as files or tutorials, discussed above. An agent has the ability to be executed 
at the user computer and perform a function such as obtaining additional information from the 
user computer system and sending the information back to the customer service rep. 

Additionally, agents can be used for many purposes. Some of the functions are 
re-configuring software, hardware or firmware; upgrading software, hardware or firmware; 
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preventive maintenance of software, hardware or firmware; fixing a failed component; updating 
and re-programming access control information. 

Preventive maintenance is a feature of the present invention's online support 
system. Preventive maintenance allows a routine executing at the user's computer to detect if a 
component is having sporadic problems and may fail in the near future. Another aspect is to 
monitor a component, such as a hard disk drive, and log the number of hours against the mean 
time to failure of the component. The component can be automatically ordered shipped to the 
user (if hardware) or automatically downloaded to the user's computer before the component 
actually fails. 

The present invention allows "remote control 1 ' of a user's computer by the 
customer service rep. Remote control allows a customer service rep to take control of a user's 
computer and control the computer by accessing applications on the user's computer from the 
customer service rep's computer. That is, commands can by typed, buttons and icons selected, 
etc., on the user's computer by having the signals from the customer service rep's input devices 
sent to the user's computer. The control can be preemptive or shared. In a shared mode both the 
user and customer service rep can control the user's computer. Voice and other data 
communication are possible during remote control. 

Thus, a system for online computer support has been discussed above. Although 
the invention has been discussed with respect to a specific embodiment, the scope of the invention 
is to be determined solely by the appended claims. 
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IN THE CLAIMS: 

1 . An online computer support system comprising: 

a user computer coupled to a communication line; 
a customer service center including a customer representative and 
electronic equipment coupled to the communication line; 

wherein the user computer and the electronic equipment each include 

means for transferring audio information over the communication 

line; 

means for transferring image information over the communication 

line; and 

means for transferring digital data over the communication line. 

2. The online computer support system of claim 1 , wherein the image information 
includes video images. 

3. The online computer support system of claim 2, wherein video images sent 
from the customer service center are played simultaneously with audio information to produce 
a real-time video including images and synchronized audio at the user computer. 



4. The online computer support system of claim 3, wherein the electronic 
equipment includes means for allowing the customer representative to annotate the video images 
with additional information. 



5. The online computer support system of claim 4, wherein the means for allowing 
the customer representative to annotate the video images comprises: 
a first display screen coupled to a mouse input device; 
a pointer displayed on the display screen in response to inputs from the mouse; 
means for detecting the mouse inputs and for moving the pointer on the display 

screen; 

wherein the user computer includes a second display screen; and 
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means for transferring information about the pointer movement at the first 
display screen over the communication line so that there is an indication of the pointer movement 
at the second display screen. 

6. The online computer support system of claim 4, wherein the means for allowing 
the customer representative to annotate the video images comprises: 

a user display screen coupled to the user computer; 
means for allowing the customer representative to access and display 
predefined information on the user display screen during presentation of the real-time video. 

7. The online computer support system of claim 5, wherein the means for 
transferring information about the pointer movement includes drawing lines on the second display 
screen in accordance with the pointer movement. 

8. The online computer support system of claim 1, wherein the user computer 
includes a display screen, wherein the image information includes an image of the customer 
representative transferred for display on the display screen, 

9. The online computer support system of claim 1, wherein the electronic 
equipment includes a display screen, wherein the image information includes an image of the user 
transferred for display on the display screen. 

10. An online computer support system comprising: 

a user computer coupled to a communication line; 

hardware components coupled to the user computer; 

a customer service center including a customer representative and 
electronic equipment coupled to the communication line; 

means, coupled to the user computer and to the electronic equipment, for 
transferring digital data over the communication line; and 

means for transferring information about the hardware from the user 
computer to the customer service center over the communication line. 
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1 1. The online computer support system of claim 10, wherein the means for 
transferring information about the hardware includes 

means for transferring an indication of the manufacturer of a given 

hardware component. 

5 12. The online computer support system of claim 10, wherein the means for 

transferring information about the hardware includes 

means for transferring an indication of the product name of a given 

hardware component. 

13. The online computer support system of claim 10, wherein a given hardware 
10 component has a version number associated with it, wherein the means for transferring 

information about the hardware includes 

means for transferring an indication of the version number of the given 

hardware component. 

14. The online computer support system of claim 10, wherein a given hardware 
15 component has a serial number associated with it, wherein the means for transferring information 

about the hardware includes 

means for transferring an indication of the serial number of the given 

hardware component. 

15. The online computer support system of claim 10, wherein a given hardware 
20 component has an installation time associated with it, wherein the means for transferring 

information about the hardware includes 

means for transferring an indication of the installation time of the given 

hardware component. 

16. The online computer support system of claim 10, wherein the user computer 
2 5 includes a processor, wherein the means for transferring information about the hardware includes 

means for transferring an indication of the processor type. 

17 
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17. The online computer support system of claim 10, wherein the user computer 
includes a processor having a speed, wherein the means for transferring information about the 
hardware includes 

means for transferring an indication of the processor speed. 

5 18. The online computer support system of claim 10, wherein the user computer 

includes a processor having a cache, wherein the means for transferring information about the 
hardware includes 

means for transferring an indication of the cache size. 

19. The online computer support system of claim 10, wherein the user computer 
10 includes a numeric co-processor, wherein the means for transferring information about the 

hardware includes 

means for transferring an indication of the numeric co-processor type. 

20. The online computer support system of claim 10, wherein the user computer 
includes a basic operating system stored in a non-volatile memory, wherein the means for 

15 transferring information about the hardware includes 

means for transferring an indication of the type of the basic operating 

system. 

21 . An online computer support system comprising: 

a user computer coupled to a communication line; 
2 0 software residing in the user computer; 

a customer service center including a customer representative and 
electronic equipment coupled to the communication line; 

means, coupled to the user computer and to the electronic equipment, for 
transferring digital data over the communication line; and 
25 means for transferring information about the software from the user 

computer to the customer service center over the communication line. 
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22. The online computer support system of claim 21, wherein the software 
includes a version number associated with the software, wherein the means for transferring 
information about the software includes 

means for transferring the version number. 

5 23. The online computer support system of claim 21, further comprising: 

hardware components coupled to the user computer; 
means for transferring information about the hardware components from 
the user computer to the customer service center over the communication line. 

10 24. The online computer support system of claim 1, further comprising: 

storage means coupled to the user computer, wherein the storage means 
includes one or more predefined specific identifiers indicating one or more aspects of the user 
computer's configuration; and 

transfer means coupled to the storage means and coupled to the 
1 5 communication line for transferring one or more of the specific identifiers from the user computer 
to the customer service center. 

25. The online computer support system of claim 24, wherein the specific 
identifiers indicate a model number of one or more components in the user computer. 

26. The online computer support system of claim 24, wherein the specific 
2 0 identifiers indicate a serial number of one or more components in the user computer. 

27. The online computer support system of claim 24, further comprising: 

a computer database located at the customer service center; 

means for receiving the specific identifiers from the user computer; and 

means for using the specific identifiers for performing a database function. 

25 28. The online computer support system of claim 27, wherein the database 

function is to perform warranty validation of the user computer. 
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29. The online computer support system of claim 27, wherein the database 
function is to perform inventory control. 

30. The online computer support system of claim 27, wherein the database 
function is to maintain a service history. 

31. The online computer support system of claim 27, wherein the database 
function is to maintain an upgrade history. 



32. The online computer support system of claim 1, further comprising: 

means for transmitting a software agent from the customer service center 
to the user computer; and 

10 means for executing the software agent on the user computer to perform 

an agent function. 

33. The online computer support system of claim 32, wherein the agent function 
is re-configuring a component of the user computer. 

34. The online computer support system of claim 32, wherein the agent function 
15 is upgrading a component of the user computer. 

35. The online computer support system of claim 32, wherein the agent function 
is performing preventive maintenance on a component of the user computer. 

36. The online computer support system of claim 32, wherein the agent function 
2 0 is fixing a component of the user computer. 

37. The online computer support system of claim 32, wherein the agent function 
is updating access control information in the user computer. 
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38. The online computer support system of claim 1, wherein one or more of the 
audio information, image information and digital data are transferred simultaneously over the 
communication line. 

39. The online computer support system of claim 1, wherein one or more of the 
audio information, image information and digital data are transferred in alternating fashion over 
the communication line. 

40. The online computer support system of claim l f wherein the user and 
customer representative communicate in a session, wherein during the session both simultaneous 
transfer and non-simultaneous transfer of different data types is used. 

41. The online computer support system of claim 40, wherein the simultaneous 
transfer is implemented using Digital Simultaneous Voice/Data (DSVD) protocol, wherein the 
non-simultaneous transfer is implemented using Asynchronous Voice/Data protocol (AVD). 



21 



WO 97/49030 



PCT/US97/09320 




WO 97/49030 



PCT/US97/09320 




FIG. 2 



WO 97/49030 



PCT/US97/09320 



3/8 



O 
O 
CVJ 



GO 
O 
C\J 



\ 



az 

LU 




CD 
O 
CM 



\ 



DC 
-JO 

can 

h- LU 

Oqc 



00 
CM 



\ 



rr Q 



CD 

cvl 



\ 



Q 
CO 

>- 

UJ 



CO 

CD 

Ll_ 




WO 97/49030 



PCT/US97/09320 



4/8 



% VoiceView TalkShop » Services 



BEE 



File Phone Options Window Help 



Please have the following informationo 
available before calling S.O.S. 

r Person Calling - 
First i . 

Name:IL_ 




e. 




Last 
Name: 



Phone 
Number: 



(■Detailed description of the problem: 



SOS Help 



Call SOS 



Cancel 



Note: If p do not have speakers and microphone attached to your PC, please connect a phone to the PC and pick 
up the receiver before pressing the *CALL SOS* buBoa 



FIG. 4 



WO 97/49030 



PCT/US97/09320 




WO 97/49030 



PCT/US97/09320 



6/8 



o 
o 



I 




WO 97/49030 



PCT/US97/09320 



7/8 

EDVoiceView Help Desk - 1 : - [Untitled]" 



HOE 



File Edit Remote Options Window Help 




ntitled] 




STATUS READY 

Macro List 



Client 



Activities 



1 04/30#6 0629PM Agent Parichay Saxena 
06:34PM Get USEBINFQ.FMP 



5ot 



06:34PM Send c:VictotiesK\se3Sion\sos8CK.vip 



3 



es 



This is the second call from this customer. He is havingj£ 
a problem with his CD caddy. 



l Save ^ 
I Close | 
I Send I 



1 Sony Call 

2 Sony Test 



3 Sony Test 



4 Intel Test 
Dir of root 
Get auto exec iconfig 
Get User Info 
Get Windows 'Wis 
Windows & System dr 



Run | | Close 



FIG. 7 



Eg]VoiceView Help Desk - 1: - fUntitledl 



HDC* 



File Edit Remote Options Window bjelp 



i«;e;;eiiei k*:ia mm 



C 1 : ^[Untitled] 



__ Eaaa| Macro List ts g 



Client 



Activities 



1 Sony Call 

2 Sony Test 



04/30/96 
04/30/96 
04/30/96 
04QPV96 



slotes 



This is t 
a problE 



f^lViewfor: 1-USERINFO.FMP 



CallersFirstName, •Parichay* 
Cal ler sLastName ' Saxena* 
Phone, •510-793-7020- 
Hardware?, '1 # 
Specific jrask? ( T 
Software?, '1' 
Upgrades?, '1* 

Problem, "The PC is not able to recognke the new Ethernet card 
Distributor, -RAB200L* 



nfig 



FIG. 8 



WO 97/49030 PCT/US97/09320 




INTERNATIONAL SEARCH REPORT 



IrrtcrnationaJ Application No. 
PCT/US97;09320 



A. CLASSIFICATION OF SUBJECT MATTER 
1PC(6) :G06P 11/00 
USCL :395/ia3J22 

According to littcrnational Patent Clarification (IPC) or to both national classification and IPC 

a FIELDS SEARCHED 

Minimum oocumentation searched (classification ayitem followed by clarification symbols) 

U.S. : 395/IB3.01, 183,07, 183.22; 1W/35 



Documentation searched other than minimum documentation to the extent that auch document! are included in the field* aearehed 



Electronic data base consulted during the international search (name of data base and, where practicable, search terms used) 
APS, STN (WPIDS, INSPEC. USPATFULL. JAPIO) 



DOCUMENTS CONSIDERED TO BE RELEVANT 



Category* 



Citation of document, with indication, where appropriate, of the relevant passages 



Relevant to claim No. 



Y 

Y,P 
Y,P 



US 5,179,695 A (DERR ET AL.) 12 January 1993, sec the abstract; 
col. 5, lines 43-63; col. 7, lines 5-29 

US 5,379,409 A (ISHIKAWA) 03 January 1995, see figure 1 

US 5,537,653 A (BIANCHIN1, JR.) 16 July 1996, see the abstract, 

US 5,621,789 A (MCCALMONT ET AL.) 15 April 1997, see the 
abstract; col. 3, lines 31-38 



Ml 

Ml 
Ml 
Ml 



| | Further documents are listed in the continuation of Box C. See patent family i 



aocaaBiot A-Trn'-i lb* p«Ml ttata of the art a 
to oa of par ti cul ar w>wmw 



*0* 



•arte doowi«u pvfelabao oo or aftat tba «rtar»aboaaJ fiting oau 

doouoaot wfckb sty throw ooubo eo priority ctaand) or wluah it 
ertad lo aatacUa* tb* yubUcauoo data of aaotbar oiwtioo or c*Wt s 
•paoial ww <aa ipostTiod) 

ooaarsaot r*ltmn« toMenl diackma*. uh. w h ihitini or otiur 
do**»Btnt »ubti*b«d prior u> (A* ot»mauor»J fUin§ d** but l**-r lhan 



■aat pubbabad aflar th* Marnatiooal filing cam or priority 
to aod not in conflict with tba application but ciud lo uooamand 
» prmcipt* or thaocy wdarrrios tba ioroBtioa 



of parbcBUr rabwanea; tha clabnso in* • noon * »pm»* b« 
novel or cannot ba coaii<W*d to involva an iovcnuv* tup 
doouaoat u takan alona 



l of oartioaiar ralavaaoa; lb* claimad invention caaoot bm 
eonsidarad lo brroita an tnvantna u*p whan lh« docuriam U 
ooaaoBMd whh ona or mora otber such documanu. »uch combination 
baieg ebvioia to a parson tkilWd m tha art 

ant mam bar of tba tuna palant fajairy 



Date of the actual completion or* the international search 
27 AUGUST 1997 


Date of mailing of the international search report 

3 1 OCT mi 


Name and mailing address of the ISA/US 
Box PCT 

Wuhingioo, D.C 20331 
Facsimile No. (703) 305-3230 


kuthorized officer 

W ALBERT DECADY ^JU 
Telephone No. (703) 30S-3900 



Form PCT/ISA/210 (second sheet)(July 1992)* 



This Page is Inserted by IFW Indexing and Scanning 
Operations and is not part of the Official Record 



Defective images within this document are accurate representations of the original 
documents submitted by the applicant. 

Defects in the images include but are not limited to the items checked: 

□ BLACK BORDERS 

□ IMAGE CUT OFF AT TOP, BOTTOM OR SIDES 

□ FADED TEXT OR DRAWING 



1ST BLURRED OR ILLEGIBLE TEXT OR DRAWING 

□ SKEWED/SLANTED IMAGES 

□ COLOR OR BLACK AND WHITE PHOTOGRAPHS 

□ GRAY SCALE DOCUMENTS 

□ LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENCE(S) OR EXHIBIT(S) SUBMITTED ARE POOR QUALITY 

□ OTHER: 

IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



BEST AVAILABLE IMAGES 




